Network storage, also referred to as network storage systems or data storage systems, is computer data storage connected to a computer network providing data access to heterogeneous clients. Typically, network storage systems process a large amount of Input/Output (IO) requests, and high availability, speed, and reliability are desirable characteristics of network storage. In addition to processing read and write IO requests, network storage systems need to perform other background operations required for maintaining these systems, such as garbage collection of obsolete data and compaction of data (e.g., merging contents of two half-filled segments into one full segment), creating snapshots of data, backup, and replication procedures, etc.
A flow of a particular processing core could be responsible for serving both read and write operations to a particular, defined portion of a storage system (e.g., LUN, volume, etc.). Generally, a flow has two separate flow scheduler queues, one for read operations, and one for write queues. In addition, in a multi-core processing environment, each processing core may handle a similar flow for accessing the same defined storage. For example, multiple flow schedulers are integrated within a corresponding flow for access to the storage, including write access flow, read access flow, etc. Each flow scheduler is configured to schedule operations from its corresponding queue of operations. However, rate limiting the entire flow across all schedulers is challenging because the queues act independently of each other, and as such no single queue has knowledge of the number of IOs being served by another queue.
What is needed are network storage device, software, and systems that provide for accounting for all IOs scheduled by all flow schedulers accessing a particular, defined portion of a storage system.
It is in this context that embodiments arise.